{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "view-in-github"
   },
   "source": [
    "<a href=\"https://colab.research.google.com/github/peremartra/Large-Language-Model-Notebooks-Course/blob/main/5-Fine%20Tuning/LoRA_Tuning_PEFT.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "j9Y40X0WCY_H"
   },
   "source": [
    "<div align=\"center\">\n",
    "<h1><a href=\"https://github.com/peremartra/Large-Language-Model-Notebooks-Course\">LLM Hands On Course</a></h1>\n",
    "    <h3>Understand And Apply Large Language Models</h3>\n",
    "    <h2>Introduction to LoRA Tuning using PEFT from Hugging Face.</h2>\n",
    "    <h3>Fine-tune a Foundational Model effortless</h3>\n",
    "    <p>by <b>Pere Martra</b></p>\n",
    "</div>\n",
    "\n",
    "<br>\n",
    "\n",
    "<div align=\"center\">\n",
    "    &nbsp;\n",
    "    <a target=\"_blank\" href=\"https://www.linkedin.com/in/pere-martra/\"><img src=\"https://img.shields.io/badge/style--5eba00.svg?label=LinkedIn&logo=linkedin&style=social\"></a>\n",
    "    \n",
    "</div>\n",
    "\n",
    "<br>\n",
    "<hr>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "bCinY2l-upqy"
   },
   "source": [
    "# LoRA Tuning\n",
    "\n",
    "In this notebook I'm introducing how to apply LoRA Tuning with the PEFT library to a pre-trained model.\n",
    "\n",
    "For a complete list of Models compatible with PEFT refer to their [documentation](https://huggingface.co/docs/peft/main/en/index#supported-methods).\n",
    "\n",
    "A short sample of models families available to be trained with PEFT are: Bloom, Llama, GPT-J, GPT-2, BERT... and more. Hugging Face is working hard to bring more Models to the Library.\n",
    "\n",
    "## Brief introduction to LoRA Tuning.\n",
    "LoRA is a re-parameterization technique. Its operation is simple, complex, and brilliant at the same time. It involves reducing the size of the matrices to be trained by dividing them in such a way that when multiplied, they yield the original matrix.\n",
    "\n",
    "The weights that are modified are those of the reduced matrices, not the original matrix. It's better visualized in an image."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "cp_7rR4Nx7My"
   },
   "source": [
    "![Matrices_LoRA.webp]()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "0oSLJkXty-t_"
   },
   "source": [
    "We have an original matrix of 50x50, which means we would have to modify about 2500 parameters. However, as we know, if we multiply two matrices of (2x50) and (50x2), we obtain a 50x50 matrix. Yet, these two matrices are formed by only 100 parameters each. In other words, for the reduced matrices, we need to modify a total of 200 parameters compared to the 2500 of the original matrix. This represents a 92% reduction, and the larger the original matrix, the greater the percentage of savings.\n",
    "\n",
    "In Language Models like GPT-3 or any of the current ones with LoRA, it's possible that we only need to train about 0.02% of the original parameters. This varies for each model. The best part is that the obtained result is very similar to that of full fine-tuning, in some cases, it can even be better."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "620MdVMk7iUS"
   },
   "source": [
    "# Load the PEFT and Datasets Libraries.\n",
    "\n",
    "The PEFT library contains the Hugging Face implementation of differente fine-tuning techniques, like LoRA Tuning.\n",
    "\n",
    "Using the Datasets library we have acces to a huge amount of Datasets."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "_UyyuMGnCPjA"
   },
   "outputs": [],
   "source": [
    "!pip install -q peft==0.8.2\n",
    "!pip install -q datasets==2.16.1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "VOnJlBY-81Wl"
   },
   "source": [
    "From the transformers library we import the necesary classes to import the model and the tokenizer.\n",
    "\n",
    "Then we can load the Tokenizer and the model.\n",
    "\n",
    "Bloom is one of the smallest and smarter model available to be trained with PEFT Library using Prompt Tuning. You can use either of the models in the Bloom Family, I encorage you to use at least two of them and see the differences.\n",
    "\n",
    "I'm using the smallest one just to spend less time trainig, and avoid memory problems in Colab."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "vziwd2UuCYGl"
   },
   "outputs": [],
   "source": [
    "from transformers import AutoModelForCausalLM, AutoTokenizer\n",
    "\n",
    "model_name = \"bigscience/bloomz-560m\"\n",
    "#model_name=\"bigscience/bloom-1b1\"\n",
    "\n",
    "tokenizer = AutoTokenizer.from_pretrained(model_name)\n",
    "foundation_model = AutoModelForCausalLM.from_pretrained(model_name)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Qtc1gbK39Hp7"
   },
   "source": [
    "## Inference with the pre-trained model.\n",
    "I'm going to do a test with the pre-trained model without fine-tuning, to see if something changes after the fine-tuning."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "jak6FzpvFTHk"
   },
   "outputs": [],
   "source": [
    "#this function returns the outputs from the model received, and inputs.\n",
    "def get_outputs(model, inputs, max_new_tokens=100):\n",
    "    outputs = model.generate(\n",
    "        input_ids=inputs[\"input_ids\"],\n",
    "        attention_mask=inputs[\"attention_mask\"],\n",
    "        max_new_tokens=max_new_tokens,\n",
    "        repetition_penalty=1.5, #Avoid repetition.\n",
    "        early_stopping=True, #The model can stop before reach the max_length\n",
    "        eos_token_id=tokenizer.eos_token_id\n",
    "    )\n",
    "    return outputs"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "GkFqjS459jAa"
   },
   "source": [
    "The dataset used for the fine-tuning contains prompts to be used with Large Language Models.\n",
    "\n",
    "I'm going to request the pre-trained model that acts like a motivational coach."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "3BAYg7czFYeK",
    "outputId": "f84ae24d-885f-4253-9554-1364514a53f9"
   },
   "outputs": [],
   "source": [
    "#Inference original model\n",
    "input_sentences = tokenizer(\"I want you to act as a motivational coach. \", return_tensors=\"pt\")\n",
    "foundational_outputs_sentence = get_outputs(foundation_model, input_sentences, max_new_tokens=50)\n",
    "\n",
    "print(tokenizer.batch_decode(foundational_outputs_sentence, skip_special_tokens=True))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "KQUGY47p9ysI"
   },
   "source": [
    "Not sure if the answer is correct or not, but for sure is not a prompt. We need to train our model if we want that acts like a prompt engineer."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "FL5L_DcR9ggA"
   },
   "source": [
    "# Preparing the Dataset.\n",
    "The Dataset used is:\n",
    "\n",
    "https://huggingface.co/datasets/fka/awesome-chatgpt-prompts"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 214,
     "referenced_widgets": [
      "86ccd10e42e047cd89c1134eb3a92abd",
      "22e5857ee96045b399f092916bf3954d",
      "06f28edecd33469cbc87c74c8233d09a",
      "8c88a9e9d9f74bd98533eb1f67ef90fb",
      "e7cd5776550c46b1b6f44c8d4ff6979d",
      "6765d5072fea4d898fc4042d588f51fa",
      "8577a237990c430d94c10b8e73b9568d",
      "71dd76f8a1484f62bea567ae8480a1b3",
      "d252c4c37cab4cf29af5c7eea047ae2f",
      "c2d9c90825f0440ebe283bcad5d58251",
      "bbba8b1a754044a7808a552bec9c344b",
      "2d93f428606e4117b7c0c17854479db9",
      "c4e2f451c3e64141b2466bc66d96dcff",
      "51bcb6d7e35d42f5916811f6344dd06e",
      "14d9cfc875104d4d95aadfbce5f7dae4",
      "ccfec8ee6e0642f2b5ec2744aebbcac8",
      "a3eaef4d0f5745e2b909222c35448851",
      "9abf798ff86e4a65bb3d332612073414",
      "88314e90d1c2453dbc881709acd9fbaf",
      "1d4661839fde481e8d0ccf274af04792",
      "a9a557a7fd914adbb563aac72ba8b1fb",
      "b54fae5c76c24f0d9371c077d1febf7b",
      "1b469fbf640c42dca8861397da1cb136",
      "8777ec7f1de64638b4dccbac9f4f4520",
      "c0cf63eee5454676a7e16c48b862a1ab",
      "8458f7a2b81f4324a74810cb159e438a",
      "a9a7f9cf9ea84da58a36dce6865e1048",
      "e934cf4bb1f8414eab2b3e34a582d661",
      "dbb61842b8b74ad8ab4e33ae54bf5074",
      "60042033fdd345558d520381fc83cb03",
      "67aa0858a779414caecc026fe452adf8",
      "30d413a93f3640f5a7a7e7d7be1bed3c",
      "0e9837d7158c40799da3a129f92883a8",
      "b984540978554516acf2b1cb3e0dc46a",
      "51aef88510e748538d81177b9f374a28",
      "36cc8f057b98482d80f4987950dd3040",
      "4c1b3dfa94124c6a856dab2ac5e86439",
      "377d46b393b34183b0e72cdbca59a4ee",
      "e661ee9a1b15404e91723a0d7f12e479",
      "20de1246fcae464d9b1e56d7949cde78",
      "1b19e705ea3f46138a442b72a9be1d5f",
      "24b0cae5949f4b088caf207542e891ab",
      "d6d84980a4a5473085eddd1bb977f8a1",
      "aa8ddb868d4242ac91134b4a2b865ce0"
     ]
    },
    "id": "DyIMQ7IHFbIx",
    "outputId": "4408edae-3196-43e9-996b-df70cb86ca05"
   },
   "outputs": [],
   "source": [
    "from datasets import load_dataset\n",
    "dataset = \"fka/awesome-chatgpt-prompts\"\n",
    "\n",
    "#Create the Dataset to create prompts.\n",
    "data = load_dataset(dataset)\n",
    "data = data.map(lambda samples: tokenizer(samples[\"prompt\"]), batched=True)\n",
    "train_sample = data[\"train\"].select(range(50))\n",
    "\n",
    "train_sample = train_sample.remove_columns('act')\n",
    "\n",
    "display(train_sample)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "gmlZY3fk_9fm",
    "outputId": "511973cb-506d-4362-e53e-01cfbcabe56c"
   },
   "outputs": [],
   "source": [
    "print(train_sample[:1])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "oVPAJsrUAHiJ"
   },
   "source": [
    "# Fine-Tuning.\n",
    "First is necesary create a LoRA config.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "bwAK6kxCDCfM"
   },
   "source": [
    "\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "uCalslQFGL7K"
   },
   "outputs": [],
   "source": [
    "# TARGET_MODULES\n",
    "# https://github.com/huggingface/peft/blob/39ef2546d5d9b8f5f8a7016ec10657887a867041/src/peft/utils/other.py#L220\n",
    "\n",
    "import peft\n",
    "from peft import LoraConfig, get_peft_model, PeftModel\n",
    "\n",
    "lora_config = LoraConfig(\n",
    "    r=4, #As bigger the R bigger the parameters to train.\n",
    "    lora_alpha=1, # a scaling factor that adjusts the magnitude of the weight matrix. Usually set to 1\n",
    "    target_modules=[\"query_key_value\"], #You can obtain a list of target modules in the URL above.\n",
    "    lora_dropout=0.05, #Helps to avoid Overfitting.\n",
    "    bias=\"lora_only\", # this specifies if the bias parameter should be trained.\n",
    "    task_type=\"CAUSAL_LM\"\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "LUddynl0B1Ck"
   },
   "source": [
    "The most important parameter is **r**, it defines how many parameters will be trained. As bigger the valuer more parameters are trained, but it means that the model will be able to learn more complicated relations between input and output.\n",
    "\n",
    "Yo can find a list of the **target_modules** available on the [Hugging Face Documentation]( https://github.com/huggingface/peft/blob/39ef2546d5d9b8f5f8a7016ec10657887a867041/src/peft/utils/other.py#L220)\n",
    "\n",
    "**lora_dropout** is like the commom dropout is used to avoid overfitting.\n",
    "\n",
    "**bias** I was hesitating if use *none* or *lora_only*. For text classification the most common value is none, and for chat or question answering, *all* or *lora_only*.\n",
    "\n",
    "**task_type**. Indicates the task the model is beign trained for. In this case, text generation.\n",
    "\n",
    "### Create the PEFT model.\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "BG1zBmhsGQ-h",
    "outputId": "32ebc3be-419c-4dab-bb3a-7b0724e1460c"
   },
   "outputs": [],
   "source": [
    "peft_model = get_peft_model(foundation_model, lora_config)\n",
    "print(peft_model.print_trainable_parameters())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "2onXUsaw-Ga0"
   },
   "source": [
    "The number of trainable parameters is really small compared with the total number of parameters in the pre-trained model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "HArPQ_lvGUkY"
   },
   "outputs": [],
   "source": [
    "#Create a directory to contain the Model\n",
    "import os\n",
    "working_dir = './'\n",
    "\n",
    "output_directory = os.path.join(working_dir, \"peft_lab_outputs\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "rWalmqWm4STo"
   },
   "source": [
    "In the TrainingArgs we inform the number of epochs we want to train, the output directory and the learning_rate."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "ND0aJ-t6ARqD"
   },
   "outputs": [],
   "source": [
    "#Creating the TrainingArgs\n",
    "import transformers\n",
    "from transformers import TrainingArguments, Trainer\n",
    "training_args = TrainingArguments(\n",
    "    output_dir=output_directory,\n",
    "    auto_find_batch_size=True, # Find a correct bvatch size that fits the size of Data.\n",
    "    learning_rate= 3e-2, # Higher learning rate than full fine-tuning.\n",
    "    num_train_epochs=2,\n",
    "    use_cpu=True\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "OgxsV-iy_J_o"
   },
   "source": [
    "Now we can train the model.\n",
    "To train the model we need:\n",
    "\n",
    "\n",
    "*   The PEFT Model.\n",
    "*   The training_args\n",
    "* The Dataset\n",
    "* The result of DataCollator, the Dataset ready to be procesed in blocks.\n",
    "\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 199
    },
    "id": "z5NYHqBnGZyF",
    "outputId": "0f756a5f-3b1a-41bc-b68d-ffef66341305"
   },
   "outputs": [],
   "source": [
    "#This cell may take up to 15 minutes to execute.\n",
    "trainer = Trainer(\n",
    "    model=peft_model,\n",
    "    args=training_args,\n",
    "    train_dataset=train_sample,\n",
    "    data_collator=transformers.DataCollatorForLanguageModeling(tokenizer, mlm=False)\n",
    ")\n",
    "trainer.train()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "kEKiFdpDGgOx"
   },
   "outputs": [],
   "source": [
    "#Save the model.\n",
    "peft_model_path = os.path.join(output_directory, f\"lora_model\")\n",
    "\n",
    "trainer.model.save_pretrained(peft_model_path)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "_TAjrSWSe14q"
   },
   "outputs": [],
   "source": [
    "#Load the Model.\n",
    "loaded_model = PeftModel.from_pretrained(foundation_model,\n",
    "                                        peft_model_path,\n",
    "                                        is_trainable=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "dK--YFPR6OxH"
   },
   "source": [
    "## Inference the fine-tuned model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "I_27uvJudf03",
    "outputId": "721df61a-199b-4494-ce81-d48ccadf6391"
   },
   "outputs": [],
   "source": [
    "input_sentences = tokenizer(\"I want you to act as a motivational coach. \", return_tensors=\"pt\")\n",
    "foundational_outputs_sentence = get_outputs(loaded_model, input_sentences, max_new_tokens=50)\n",
    "\n",
    "print(tokenizer.batch_decode(foundational_outputs_sentence, skip_special_tokens=True))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "xCV9JOBG6Ug8"
   },
   "source": [
    "The result is amazing. Let's compare the answer of the pre-trained Model withe the one fine-tuned by us using LoRA:\n",
    "* **Pretrained Model:** *I want you to act as a motivational coach.*  Don't be afraid of being challenged.\n",
    "* **Fine-Tuned Model:** I want you to act as a motivational coach.  I will provide some information about someone\\'s motivation and goals, but it should be your job  in order my first request – \"I need someone who can help me find the best way for myself stay motivated when competing against others.\" My suggestion is “I have\n",
    "\n",
    "As you can see the result is really similar to the samples containmed in the Datased used to fine-tune the Model. And we only trained the Model for 10 epochs and with a really small number of rows.\n",
    "\n",
    "# Continue Learning\n",
    "Please, play with all the variables in the notebook and drive your own experiments and get your conclusions.\n",
    "\n",
    "Try to change the **lora_config** values, maybe you can achieve a better result in less epochs, saving time and money for your company. :-)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "tr4Sm32y89Ji"
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "colab": {
   "authorship_tag": "ABX9TyNBihjGx70DtC3db82s+h3x",
   "include_colab_link": true,
   "provenance": []
  },
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.13"
  },
  "widgets": {
   "application/vnd.jupyter.widget-state+json": {
    "06f28edecd33469cbc87c74c8233d09a": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_71dd76f8a1484f62bea567ae8480a1b3",
      "max": 274,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_d252c4c37cab4cf29af5c7eea047ae2f",
      "value": 274
     }
    },
    "0e9837d7158c40799da3a129f92883a8": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "14d9cfc875104d4d95aadfbce5f7dae4": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_a9a557a7fd914adbb563aac72ba8b1fb",
      "placeholder": "​",
      "style": "IPY_MODEL_b54fae5c76c24f0d9371c077d1febf7b",
      "value": " 74.6k/74.6k [00:00&lt;00:00, 363kB/s]"
     }
    },
    "1b19e705ea3f46138a442b72a9be1d5f": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "1b469fbf640c42dca8861397da1cb136": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_8777ec7f1de64638b4dccbac9f4f4520",
       "IPY_MODEL_c0cf63eee5454676a7e16c48b862a1ab",
       "IPY_MODEL_8458f7a2b81f4324a74810cb159e438a"
      ],
      "layout": "IPY_MODEL_a9a7f9cf9ea84da58a36dce6865e1048"
     }
    },
    "1d4661839fde481e8d0ccf274af04792": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "20de1246fcae464d9b1e56d7949cde78": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "22e5857ee96045b399f092916bf3954d": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_6765d5072fea4d898fc4042d588f51fa",
      "placeholder": "​",
      "style": "IPY_MODEL_8577a237990c430d94c10b8e73b9568d",
      "value": "Downloading readme: 100%"
     }
    },
    "24b0cae5949f4b088caf207542e891ab": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "2d93f428606e4117b7c0c17854479db9": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_c4e2f451c3e64141b2466bc66d96dcff",
       "IPY_MODEL_51bcb6d7e35d42f5916811f6344dd06e",
       "IPY_MODEL_14d9cfc875104d4d95aadfbce5f7dae4"
      ],
      "layout": "IPY_MODEL_ccfec8ee6e0642f2b5ec2744aebbcac8"
     }
    },
    "30d413a93f3640f5a7a7e7d7be1bed3c": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "36cc8f057b98482d80f4987950dd3040": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_1b19e705ea3f46138a442b72a9be1d5f",
      "max": 153,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_24b0cae5949f4b088caf207542e891ab",
      "value": 153
     }
    },
    "377d46b393b34183b0e72cdbca59a4ee": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "4c1b3dfa94124c6a856dab2ac5e86439": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_d6d84980a4a5473085eddd1bb977f8a1",
      "placeholder": "​",
      "style": "IPY_MODEL_aa8ddb868d4242ac91134b4a2b865ce0",
      "value": " 153/153 [00:00&lt;00:00, 624.89 examples/s]"
     }
    },
    "51aef88510e748538d81177b9f374a28": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_e661ee9a1b15404e91723a0d7f12e479",
      "placeholder": "​",
      "style": "IPY_MODEL_20de1246fcae464d9b1e56d7949cde78",
      "value": "Map: 100%"
     }
    },
    "51bcb6d7e35d42f5916811f6344dd06e": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_88314e90d1c2453dbc881709acd9fbaf",
      "max": 74565,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_1d4661839fde481e8d0ccf274af04792",
      "value": 74565
     }
    },
    "60042033fdd345558d520381fc83cb03": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": "20px"
     }
    },
    "6765d5072fea4d898fc4042d588f51fa": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "67aa0858a779414caecc026fe452adf8": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "71dd76f8a1484f62bea567ae8480a1b3": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "8458f7a2b81f4324a74810cb159e438a": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_30d413a93f3640f5a7a7e7d7be1bed3c",
      "placeholder": "​",
      "style": "IPY_MODEL_0e9837d7158c40799da3a129f92883a8",
      "value": " 153/0 [00:00&lt;00:00, 567.98 examples/s]"
     }
    },
    "8577a237990c430d94c10b8e73b9568d": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "86ccd10e42e047cd89c1134eb3a92abd": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_22e5857ee96045b399f092916bf3954d",
       "IPY_MODEL_06f28edecd33469cbc87c74c8233d09a",
       "IPY_MODEL_8c88a9e9d9f74bd98533eb1f67ef90fb"
      ],
      "layout": "IPY_MODEL_e7cd5776550c46b1b6f44c8d4ff6979d"
     }
    },
    "8777ec7f1de64638b4dccbac9f4f4520": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_e934cf4bb1f8414eab2b3e34a582d661",
      "placeholder": "​",
      "style": "IPY_MODEL_dbb61842b8b74ad8ab4e33ae54bf5074",
      "value": "Generating train split: "
     }
    },
    "88314e90d1c2453dbc881709acd9fbaf": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "8c88a9e9d9f74bd98533eb1f67ef90fb": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_c2d9c90825f0440ebe283bcad5d58251",
      "placeholder": "​",
      "style": "IPY_MODEL_bbba8b1a754044a7808a552bec9c344b",
      "value": " 274/274 [00:00&lt;00:00, 6.22kB/s]"
     }
    },
    "9abf798ff86e4a65bb3d332612073414": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "a3eaef4d0f5745e2b909222c35448851": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "a9a557a7fd914adbb563aac72ba8b1fb": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "a9a7f9cf9ea84da58a36dce6865e1048": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "aa8ddb868d4242ac91134b4a2b865ce0": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "b54fae5c76c24f0d9371c077d1febf7b": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "b984540978554516acf2b1cb3e0dc46a": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_51aef88510e748538d81177b9f374a28",
       "IPY_MODEL_36cc8f057b98482d80f4987950dd3040",
       "IPY_MODEL_4c1b3dfa94124c6a856dab2ac5e86439"
      ],
      "layout": "IPY_MODEL_377d46b393b34183b0e72cdbca59a4ee"
     }
    },
    "bbba8b1a754044a7808a552bec9c344b": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "c0cf63eee5454676a7e16c48b862a1ab": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_60042033fdd345558d520381fc83cb03",
      "max": 1,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_67aa0858a779414caecc026fe452adf8",
      "value": 1
     }
    },
    "c2d9c90825f0440ebe283bcad5d58251": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "c4e2f451c3e64141b2466bc66d96dcff": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_a3eaef4d0f5745e2b909222c35448851",
      "placeholder": "​",
      "style": "IPY_MODEL_9abf798ff86e4a65bb3d332612073414",
      "value": "Downloading data: 100%"
     }
    },
    "ccfec8ee6e0642f2b5ec2744aebbcac8": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "d252c4c37cab4cf29af5c7eea047ae2f": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "d6d84980a4a5473085eddd1bb977f8a1": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "dbb61842b8b74ad8ab4e33ae54bf5074": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "e661ee9a1b15404e91723a0d7f12e479": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "e7cd5776550c46b1b6f44c8d4ff6979d": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "e934cf4bb1f8414eab2b3e34a582d661": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    }
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
